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Abstract 

We consider the use of energy harvesters, in place of conventional batteries with fixed energy storage, for 
point-to-point wireless communications. In addition to the challenge of transmitting in a channel with time selective 
fading, energy harvesters provide a perpetual but unreliable energy source. In this paper, we consider the problem 
of energy allocation over a finite horizon, taking into account channel conditions and energy sources that are time 
varying, so as to maximize the throughput. Two types of side information (SI) on the channel conditions and 
harvested energy are assumed to be available: causal SI (of the past and present slots) or full SI (of the past, 
present and future slots). We obtain structural results for the optimal energy allocation, via the use of dynamic 
programming and convex optimization techniques. In particular, if unlimited energy can be stored in the battery 
with harvested energy and the full SI is available, we prove the optimality of a water-filling energy allocation 
solution where the so-called water levels follow a staircase function. 
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I. Introduction 

In conventional wireless communication systems, the communication devices have access to a fixed 
power supply, or are powered by replaceable or rechargeable batteries to enable user mobility. In these 
cases, the transmissions are limited by power constraints for safety reasons, or by the sum energy constraint 
so as to prolong operating time for battery-powered devices. In other communication systems, however, 
a fixed power supply is not readily available, and even replacing the batteries periodically may not be 
a viable option if the replacement is considered to be too inconvenient (when thousands of senor nodes 
are scattered throughout the building), too dangerous (the devices may be located in toxic environments) 
or even impossible (when the devices are embedded in building structures or inside human bodies). In 
such situations, the use of energy harvesting for wireless communications appears appealing or sometimes 
even essential. Examples of energy that can be harvested include solar energy, piezoelectric energy and 
thermal energy, etc. 

For transmitters that are powered by energy harvesters, the energy that can potentially be harvested is 
unlimited. Typically, energy is replenished by the energy harvester, while expended for communications 
or other processing; any unused energy is then stored in an energy storage, such as a rechargeable battery. 
However, unlike conventional communication devices that are subject only to a power constraint or a sum 
energy constraint, transmitters with energy harvesting capabilities are, in addition, subject to other energy 
harvesting constraints. Specifically, in every time slot, each transmitter is constrained to use at most the 
amount of stored energy currently available, although more energy may become available in the future 
slots. Thus, a causality constraint is imposed on the use of the harvested energy. 

Several contributions in the literature have considered using energy harvester as an energy source, in 
particular based on the technique of dynamic programming [0Q|. hi 0, the problem of maximizing a 
reward that is linear with the energy used is studied. In |fjQ, the discounted throughput is maximized over 
an infinite horizon, where queuing for data is also considered. In 0], adaptive duty cycling is employed for 
throughput maximization and implemented in practical systems. In [[5]|, an information-theoretic approach 
is considered where the energy is harvested at the level of channel uses. In [O, and the references therein, 
optimal approaches are also considered for throughput maximization over AWGN or fading channels. 

In this work, we consider the problem of maximizing the throughput via energy allocation over a 
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finite horizon of K time slots. The channel signal-to-noise ratios (SNRs) and the amount of energy 
harvested change over different slots. Our aim is to study the structure of the maximum throughput and 
the corresponding optimal energy allocation solution, such as concavity and monotonicity. These results 
may be useful for developing heuristic solutions, since the optimal solutions are often complex to obtain 
in practice. We consider two types of side information (SI) available to the transmitter: 

• causal SI, consisting of past and present channel conditions, in terms of SNR, and the amount of 
energy harvested in the past slots, or 

• full SI, consisting of past, present and future channel conditions and amount of energy harvested. 
The case of full SI may be justified if the environment is highly predictable, e.g., the energy is harvested 
from the vibration of motors that are turned on only during fixed operating hours and line-of-sight is 
available for communications. 

Our contributions are as follows. Given causal SI, and assuming that the variations in the channel 
conditions and energy harvested are modeled by a first-order Markov process, we obtain the optimal energy 
allocation solution by dynamic programming, which can be computed offline and stored in a lookup table 
for implementation. Moreover, we obtain structural results to characterize the optimal solution. Given full 
SI, we obtain a closed-form solution for K = 2 slots. We also obtain the structure of this optimal solution 
for arbitrary K with unlimited energy storage. The optimal solution then has a water-filling interpretation, 
as in 0. However, instead of a single water level, there are multiple so-called water levels that are non- 
decreasing over time, i.e., the water levels follow a staircase-like function. Finally, we propose a heuristic 
scheme that uses only causal SI. Compared to a naive scheme, the proposed scheme performs relatively 
close to the optimal throughput obtained with full SI in our numerical studies. 

This paper is organized as follows. Section [I] gives the system model. Then, Section [III] considers 
optimal schemes with availability of causal SI of the channel conditions and harvested energy. Section [IV] 
considers optimal schemes with availability of full SI with a constraint on the maximum amount of 
energy that can be stored on the battery, while Section IVl considers the specific case where this constraint 
is removed. Section [VI] shows numerical results for the various schemes. Finally, Section IVTll concludes 
the paper. 
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Fig. 1. Block diagram of a transmitter powered by an energy harvester. Energy is replenished by an energy harvester but is drawn for 
transmission. 



II. System Model 

For simplicity, each packet transmission is performed in one time slot. Each time slot allows n symbols 
to be transmitted, where n is assumed to be sufficiently large for reliable decoding. We index time by 
the slot index k e K = {1, • • ■ , K}. We assume that there is always back-logged data available for 
transmission. In slot k, a message Wk G {1, • • • , 2 nRk } is sent, where the rate R k > in bits per symbol 
can be selected. 

We consider a point-to-point, flat-fading, single-antenna communication system. As shown in Fig. [TJ 
the transmitter is powered by an energy harvester that takes in harvested energy as input, that is then 
stored in an energy storage. The bits to be sent are encoded by an encoder, then sent by a power amplifier 
that uses the stored energy. Energy is measured on a per symbol (or channel use) basis, hence we use the 
terms energy and power interchangeably. 

Consider slot k & K,. At time instant k~, which denotes the time instant just before slot k, the battery 
has available > amount of stored energy per symbol. For transmission, the message Wk is first 
encoded as data symbols XJ! = [X±k, • • • ,X n k] of length n, where we normalize Y17=i \Xik\ 2 /n = 1. 
Then the transmitter transmits packet k in slot k as s/TkX^, where < T k < B k is the energy per symbol 
used by the power amplifier. Except for transmission, we assume the other circuits in the transmitter 
consume negligible energy. 

1) Mutual Information: We assume the channel is quasi-static for every slot k E K with SNR jk. The 
maximum reliable transmission rate in slot k is then given by the mutual information I^^Tk) > in 
bits per symbol (8). In general, we assume that 1(7, T) is concave in T given 7, and is increasing in T 
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for all 7. For example, we may employ Gaussian signalling for transmission over a complex Gaussian 
channel [8], which gives 

/( 7 ,T) = log 2 (l + T 7 ). (1) 

2) Battery Dynamics: In general, let us denote a vector of length k as Y k = [Y 1 , ■ ■ ■ ,Y k ], e.g., the 
battery energy from slot 1 to slot k is given by B k = [Bi, ■ ■ ■ , B k ]. While transmitting packet k, the 
energy harvester collects an average energy of H k > per symbol, which is then stored in the battery. 
At time instant (k + 1)~, the energy stored is updated in general as 

B k+1 = f(B k ,T k ,H k ),ke)C, 

where the function / depends on the battery dynamics, such as the storage efficiency and memory effects. 
Intuitively, we expect B k+1 to increase (or remains the same) if B k or H k increases, or if T k decreases. As 
a good approximation in practice, we assume the stored energy increases and decreases linearly provided 
the maximum stored energy in the battery -B max is not exceeded, i.e., 

B k+1 = mm{B k -T k + H k , B max }, k e JC. (2) 

We assume the initial stored energy B\ is known, where < B\ < B max . Thus, {B k } follows a 
deterministic first-order Markov model that depends only on the immediate past random variables. 

3) Channel and Harvest Dynamics: To model the unpredictable nature of energy harvesting and the 
wireless channel over time, we modeJ3 H K ^ and 7 A jointly as a random process described by their 
joint distribution. The exact distribution depends on the energy harvester used and the wireless channel 
environment. 

In most typical operating scenarios, both the wireless channels and the harvested energy vary slowly 
over time. To account for these variations, the SNR 7^ is assumed to be constant in each slot and follow 
a first-order stationary Markov model over time k, see e.g. similar assumptions in [0. Also, the harvested 
energy H k is modeled as first-order stationary Markov model over time k, where the accuracy of this 
model is justified by empirical studies when solar energy is harvested ifTOl . Given H = H and 71 = 71, 

'The harvested energy in slot K, namely Hk, cannot be used for transmission in slots 1 to slot K and so does not affect the throughput. 



the joint pdf of H and 7 thus becomes 



Ph k-,,k{H k -\ 1 k \H q = #0,71 = Ti) 



K 



=11 PH k-i (H k ~i\H k _ 2 )p lk (7fc|7fc_i) 



k=3 

xp Hl (Hi\H = #o)p 7 2(72|7i = 7i) (3) 

where p# fe (-|-) and p 7k (-\-) are independent of fc. In ©, we have also assumed that the harvested energy 
and the SNR are independent, which is reasonable in most practical scenarios. In this paper, we assume 
that the joint distribution ® is known, which may be obtained via long-term measurements in practice. 

4) Overall Dynamics: Let us denote the state s k = (7*, H k -i, B k ), k G /C, or simply s if the index k 
is arbitrary. Let the accumulated states be s fc = (si, ■ • • , S&), k E JC. 

We assume the initial state Sj — (7 1 ,# ,i? 1 ) to be always known at the transmitter, which may be 
obtained causally prior to any transmission. From © and (|3]), given si = s l5 the states thus follow a 
first-order Markov model: 

K 



P s k(s k \s 1 = si) = JJp Sfc (s fc |s fc _!) x p S2 (s 2 |s! = Si). (4) 

fc=3 

In particular, © includes the special cases where the states are independent, i.e., p Sfe (sfc|sA;_i) = p Sk (sk), 
or where the states are deterministic rather than random, i.e., p Sfe (s^ |s fc _i) = <5(s fc — s k ), where S(-) is the 
Dirac delta function. 

In the next three sections, we consider the problem of maximizing the throughput subject to energy 
harvesting constraints, given either causal SI or full SI. 

III. Causal Side Information 

A. Problem Statement 

We first consider the case of causal SI, in which the transmitter is given knowledgdj of Sfc before 
packet k is transmitted, where k e JC. That is, at slot k the transmitter only knows the present channel 
SNR 7fe, past harvested energy H k and present energy stored in the battery B k . In practice, for instance, 
the receiver feeds back 7^ shortly before transmission, while the transmitter infers Hk-i and B k from its 

2 It can be shown that having knowledge of previous states s k ~ 1 does not improve throughput, due to the Markovian property of the states 
in {4}. 



energy storage device. We say that causal SI is available as future states are not a priori known. Thus, this 
allows us to model and treat the unpredictable nature of the wireless channel and harvesting environment. 

The causal SI is used to decide the amount of energy T k for transmitting packet k. We want to maximize 
the throughput, i.e., the expected mutual information summed over a finite horizon of K time slots, by 
choosing a deterministic power allocation policy ir = {Tk(sk), Vsfc, k = 1, • • • , K}. The policy can be 
optimized offline and implemented in real time via a lookup table that is stored at the transmitter. 

A policy is feasible if the energy harvesting constraints < T k (s k ) < B k is satisfied for all possible s k 
and all k e /C; we denote the space of all feasible policies as II. Mathematically, given s l5 the maximum 
throughput is 

T* = max7», (5) 
Tren 

where 

K 

r(7r) = ^E[/( 7fc ,T fc (s fc ))|s 1 ,7r]. (6) 
fc=i 

In ©, the A;th summation term represents the throughput of packet k (after expectation); its expectation 
is performed over all (relevant) random variables given initial state s\ and policy n. 
For example, if K = 2 and a given policy, © simplifies as 



T=/(7i,7\(si))+E S2 /( 7 2,T 2 (s 2 )) 



si 



(7) 



subject to < Ti < B 1 for the first term and < T 2 < B 2 = min{i?i - T x + Hi, B m3uX } for the second 
term. Clearly, the transmission energy T x in the first slot affects the stored energy B 2 available in the 
second slot, which in turn affects the energy T 2 to be allocated. 

In general the optimization of {T k } cannot be performed independently due to the energy harvesting 
constraints, as shown also in the above example. Instead, for the above example, we can first optimize T 2 
given all possible 7\ (and hence all possible B 2 ), then optimize for T x with T 2 replaced by the optimized 
value (as a function of T{). This approach, as will be suggested by dynamic programming in the general 
case, will be shown to be optimal. 



B. Optimal Solution 

The optimization problem © is solved by dynamic programming in Lemma \T\ 



Lemma 1: Given initial state s x = (ji,H ,Bi), the maximum throughput T* is given by Ji(si), which 
can be computed recursively based on Bellman's equations, starting from Jk($k), Jk-i{sk-i), an d so 
on until Ji(si): 

J K (j,H,B)= max . I( 7 ,T) = I(j,B), (8a) 
J fc ( 7 , if, 5) = max I( 7 , T) + J fe+1 ( 7 , H,B — T) (8b) 

for fc = 1, • • ■ , if — 1, where 



Efr - 

Ha 



J fc+ i( 7 ,ii,min{ J B max ,x + #}) | 7 , iJ 



(9) 



In ©, if denotes the harvested energy in the present slot given the harvested energy H in the past slot, 
and 7 denotes the SNR in the next slot given the SNR 7 in the present slot. An optimal policy is denoted 
as tv* = {T fc *(s fc ), Vs fc , k = 1, • • • , if}, where T fc *(s fc ) is the optimal T that solves ©. 

Proof: The proof follows by applying Bellman's equation [OQ and using © and ©. ■ 
In (l8al) . the optimal maximization is trivial: the interpretation is that we use all available energy for 
transmission in slot K. We can interpret the maximization in (l8bl as a tradeoff between the present and 
future rewards. This is because the mutual information /(■,•) represents the present reward, while Jk+i, 
commonly known as the value function, is the expected future mutual information accumulated from slot 
k + 1 until slot K. 

Next, we obtain structural properties of the maximum throughput T* in © and the corresponding 
optimal policy n* in Theorems \T\ and |2l The proofs are given in the Appendix. 
Theorem 1: Suppose that J( 7 ,T) is concave in T given 7. Given 7 and H, then 

1) Jfc(7, H, B) in © is concave in B for k G /C; 

2) </fc(7, H, B) in © is concave in B for k E )C. 
Thus, T* = Ji(sx) is concave in B 1 . 

Theorem 2: Suppose that 1(7, T) is concave in T given 7. Given 7 and if, then the optimal power 
allocation T£( 7 , if, S) that solves © is non-decreasing in B, where k E fC. 

The structural properties in Theorems [TJ and |2] simplify the numerical computation of the optimal power 
allocation solution in Lemma [Q as shown in the next subsection. 



C. Numerical Computations 

From (|8a1 ), we get the optimal solution for slot K as T^(s^) = B K . Now, consider the problem of 
finding the optimal T k (s k ) to obtain J k (s k ),k e {1, • • • , K — 1}. Let us fix the SNR and harvested 
energy as 7, if, respectively, and drop these arguments when possible to simplify notations. Consider the 
unconstrained maximization over all T > 0, i.e., not subject to any energy harvesting constraint: 

Tl = argmax^(T) (10) 

where we denote g(T) = 1(7, T) + J k+ i(B — T). Since 1(7, T) is concave, and J k+ i{B — T) is concave 
due to Theorem \T\ the objective function g{T) is concave. Thus, the maximization over all T gives 
a unique solution T k , easily solved using numerical techniques such as a bisection search ifTD . Also, 
Theorem |2] helps to reduce the search space by restricting the search to be in one direction for different 
B. Alternatively, if g(T) is differentiable and available in closed-form, T k is given by solving g'(T) = 0. 
Finally, we get the optimal solution for (l8bl) by restricting the maximization in (PTOl) to be over < T < B 
to give 

0, Tl < 0; 



n 



B, Tl>B; (11) 



4 0<T fc t <R 

This is because if < 0, the (concave) objective function g(T) must be decreasing for T > 0; if T\ > B, 
the objective function must be increasing for T < B. 

D. I.I.D. SNR and Harvested Energy 

We consider the Ltd. SI scenario where both 7^ and H k are independent and identically distributed 
(i.i.d.) over k for analytical tractability. Even with i.i.d. SI, the optimization problem in Lemma [T] is not 
decoupled as it still depends on the past harvested energy H k _i. Intuitively, this is because the present 
transmission energy T k (whose maximum allowable depends on H k _i) will still affect the future storage 
energy B k+1 , B k+2 , ■ ■ ■ . 

If we assume a Rayleigh fading channel with expected SNR given by 7, i.e., the statistics of the SNR 
is ^7(7) = l/7exp(— 7/7), 7 > 0, the expected mutual information evaluates as 

7(T) 4 E 7 [/( 7 ,T)] = exp ( -±=) E x ( (12) 



7T / V7T 
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where the exponential integral is defined as Ei (x) = exp(— t)/t dt. Instead, if we assume an AWGN 
channel where the channel is time-invariant with 7^ = 7 for all k, then the expected mutual information 
is simply 

J(T) = /(7,T)=log(l+7T). (13) 

In AWGN channels, by inspection Jkij, H, B) in Lemma[T]is independent of 7 for all k, but still dependent 
on H. Hence, the optimization problem for each J* (7, H, B) still has to be solved recursively, rather than 
as decoupled optimization problems. 

IV. Full Side Information: Arbitrary 5 max 

The initial battery energy Bi is always known by the transmitter. We say that full SI is available if the 
transmitter also has priori knowledge of the harvest power H K ~ l and SNR 7 A before any transmission 
begins. This corresponds to the ideal case of a predictable environment where the harvest power and 
channel SNR are both known in advance, and also gives an upper bound to the maximum throughput T* 
for any distribution d3j. 

In this section, we consider the general case where B mSLX may be finite. Corollary Q] as a consequence 
of Lemma [Q gives the optimal throughput T* for the same problem © but with full SI available. 
Corollary 1: Given full SI {H K ~ l ,^ K }, the maximum throughput is given by 



K 



J 1 ( J B 1 ) = maxV/(7 fc ,T fc ), (14) 
k=l 

which can be computed recursively based on Bellman's equations: 

J K {B) = max J( 7K ,T) = I(>y K ,B), (15a) 
J k {B) 

= max 7( Tfe) T) + J k+1 (mm{B nmx , B-T + H k }) (15b) 

0<T<B 

for k = 1,-- ■ ,K- 1. 

Proof: All side information are a priori known and hence the SI is deterministic rather than random. 
Corollary [ijthus follows immediately from Lemma [T] by replacing the pdfs in © by Dirac delta functions 
accordingly. ■ 
In general, power may be allocated via these modes: 
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• greedy (G): use all stored energy whenever available; 

• conservative (C): save as much stored energy as possible (without wasting any harvested energy) to 
the last slot; 

• balanced (B): stored energy is traded among slots accordingly to channel conditions. 

For the last slot, or if K = 1 where there is only one slot, from (Q3a) it is optimal to allocate all power 
for transmission. For the case K = 2, Corollary |2] obtains the optimal power allocation for the first slot. 
The proof is given in the Appendix. 

Corollary 2: Consider K = 2 slots. Suppose that the mutual information function is given by ©. 
Given full SI {Bi, Hi, ji, 72}, the optimal transmission energy for slot 1 is given by (corresponding to 
the G, B, C modes, respectively) 

B 1 , a < or B 1 < b; 

f, a > and - b < B 1 < c; (16) 

[B 1 - a] + , a > and (B 1 > c or B 1 < -b); 
where we denote [x] + = max(0,x) and 

f = B,/2 + (l/ 7a - l/ 7l + #0/2, (17) 

and we also let a = B mSLX - E x , b = H 1 + I/72 - I/71, and c = 2 J B max - #1 + I/72 - l/7i- 

In Corollary |2] the power allocation ([TBI is interpreted to be in G, B, or C mode, respectively. As an 
example, suppose b > 0. Then all modes can be active: power allocation is greedy if the energy to be 
harvested is large or the stored energy is small (a < or B\ < b); power allocation is conservative if 
the energy to be harvested is small and the stored energy is large (a > and B\ > c); otherwise, the 
allocation depends on the SI. 

Remark 1: From Corollary |2] T*(Bi) is a piece-wise linear function of B 1 . We also see that T*(Bx) 
is increasing in B 1 , as stated in Theorem |2] for the general case. 

Remark 2: If B max — > 00, we get a > and c — > 00 in Corollary |2] Then ([TBI simplifies to T* = [T] + . 
From (fTTl , the optimal power allocation is thus given by half of the battery energy B 1 /2 plus (or minus) 
a correction term that depends on the SNRs and harvested energy; this observation will be exploited to 
obtain a heuristic scheme in Section IVI-AI 
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Although we can derive a closed-form result for larger K, the expression becomes unwieldy and less 
intuitive. To make progress, in the next section we assume the case of infinite B max , which gives the highest 
possible achievable throughput and thus provides an upper bound for any practical implementation. The 
assumption is also reasonable if the storage buffer is selected to be large enough. We shall show that for 
any K we can obtain a closed-form result that is a variation of the water-filling power allocation policy 
0, which is somewhat suggested by Remark [21 

V. Full Side Information: Infinite B mSLX 

The previous section considers the general case of arbitrary B max . To develop more insights, in this 
section we consider that the mutual information function is given by (OQ) and B max — > oo. Then from ©, 
the battery stored at slot k + 1, where k E /C, is given by 

k k 

B k+1 = B 1 -^2T i + Y i H i . (18) 

i=l i=l 

A non-negative power allocation is feasible if and only if B k+1 > 0, k E /C. The throughput maximization 
problem solved in Corollary [T]can then be formulated as follows: 

K 

T*= max Vl( 7fc ,T fc ) (19a) 

{T fc >0,fcG/C}f-f 
k=l 

k k-1 

subject to T * - B i ~ Yl Hi - °' kelC - (19b) 

i=l i=l 

A. Water-Filling Algorithm 

Before we consider the general case where the constraint (|19b| ) is imposed for all k E /C, we impose 
the constraint (1 1 9bb only for the last slot, i.e., only for k = K. This then corresponds to the conventional 
problem of maximizing the sum throughput with a sum energy constraint of P max = B\ + ^Y^i=i Hi : 

K 

r^ F ( 7 K ,P m ax)= r max V/( Tfc) T fe ) (20a) 

K 



Subject tO Y^ Ti < -Pmax- (20b) 



1=1 



Since less constraints are imposed, the maximum throughput in (|20l ) is no smaller than that of (fT9l . It is 
well known that the optimal solution for (|20l ) is given by (see e.g. (H, IfTTIO 

1 



rp-k 

1 WF,i 



V 

Ik 



(21) 
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This optimal solution is implemented by the water-filling algorithm, where the water-level (WL) v > is 
chosen such that (I20bl) holds with equality by using the optimal power allocation in (12TI) . For completeness, 
an implementation of the water-filling algorithm, which gives the maximum T to within a tolerance of 
e, is given below as Algorithm \T\ 

input : slot size K, SNRs {7^}, power constraint P max , tolerance e (close to zero) 
output: optimal power allocation {T^ ¥k \, optimal WL A* 

// initialization 

P := 0, A lD := 0, A hi := 00 (a large number) ; 

T e k := [\ e -l/ 7fc ]+ 0e{lo,hi},fce/C ; 

II loop until sum power P less than P max to within tolerance e 
while |P max - P\ > e or P > P max do 

/ / improve P to be closer to P max 

A := (A lG + A hi )/2 ; 

T k := [A-l/ 7fc ]+ fce/C ; 

p '■= Ylk=i T k> 

1 1 update Aio or A^ 
if P > P max then 
A hi := A; 

else 

A lD = A; 

end 

end 

^WF,fc := Tk', 

X* := A; 

Algorithm 1: Conventional water- filling algorithm. This implementation achieves optimality to a 
tolerance of e. 
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B. Staircase Water- Filling Algorithm 

We now proceed to solve our original problem ([19} with additional energy harvesting constraints in 
(|19b| ). It turns out that the conventional water-filling algorithm is no longer optimal. Instead it is necessary 
to use a generalized type of water-filling where the water level is a staircase-like function. 

1 ) Structural Properties: The optimization problem in (fT9l is convex and so can be solved by the dual 
problem [[TTfl . The Lagrangian associated to the primal problem (fT9l is 

K / k fc-1 

C(X K , T K ) = T(T K ) -J2x k -[J2 T i- B i-J2 H i 

k=l \i=l t=l 

where T k > is the power allocation for the kth slot and A^ > is the Lagrangian multiplier for the kth 
constraint in (|19b| ), k E fC. Then the necessary and sufficient conditions for A^ and T K to be both primal 
and dual optimal are given by the Karush-Kuhn-Tucker (KKT) optimality conditions: 

k fc-1 

^2T i -B 1 -^H i <0, (22a) 

i=l 1=1 

T k > 0, (22b) 
A fc > 0, (22c) 

(fc fc-1 \ 

^Ti-Bt-^HA =0, (22d) 
i=i i=i / 

ffi^ = 0, (22e) 
oT k 

for k E K. From (I22bl) . and imposing the constraints (122cb and (I22el ) via similar arguments to obtain (|2~TT> 
in [[81, [fTTI . we obtain the optimal power allocation as 



i 

^ — 

Ik. 



(23) 



for k E K, where v k = ^ln2 5^^ fe A^ > 0, and the A/s satisfy the KKT conditions ( f22l) . 

Analogous to the problem in (|20T ) with only power constraint (120bl) . we say z/^ is the WL for slot k. 
Also, we say slot t E /C is a transition slot (TS) if the water level changes a/ter slot t, i.e., 7^ We 
define the last slot k = K also as a TS (say by defining vk+x t0 be infinity); hence there is at least one 
TS. We collect all TSs as the set S = {t l5 1 2 , ■ • • >i|s|}> where U < tj for i < j and t\s\ = K. 

From the result in (l23l . we obtain the following structural properties for the optimal power allocation 
in Theorem |3l Fig. |2] gives an example of the optimal power allocation. In general the optimal WLs {u k } 
depend on the slot indices and there can be multiple optimal TSs, while in the conventional water- filling 
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j i slot k 
1 : = ► 

h t 2 

Fig. 2. Structure of optimal power allocation T£ with full SI and infinite -B max - We assume two optimal TSs and hence three distinct water 
levels for Uk- Here, the SNRs {7fe} are arbitrary. 

algorithm, the optimal WL is the same for all slot indices and thus there is no TS (except for the trivial 
one at slot K). 

Theorem 3: The optimal power allocation in (1231 ) satisfy these properties: 
PI: The WL is non-decreasing over slots, i.e., v\ < ■ ■ • < uk- We say that the optimal power allocation 
performs staircase water-filling over slots, since the WL is a staircase-like function (see e.g. Fig. [2]). 
P2: If slot t is a TS, then the battery storage is empty, i.e., (122al ) holds with equality if k G S. 

Proof: Since \ k > 0, it follows that v k > and also that v k is non-decreasing with k. This proves 
property PI. 

Suppose slot t is an TS, i.e., t E S and so v t ^ vt+i- Since by definition v k = (In 2 Ylf=k ^t) -1 > we S et 
At ^ 0. From (I22cl) . we get \ t > 0. It then follows from the complementary slackness condition (I22dl) 
(with k replaced by t) that (I22al) holds with equality for k = t. This proves property P2. ■ 
From Theorem |3] we have the following additional structural properties in Corollary |3]and Corollary @] 
Corollary 3: If the SNR is non-decreasing over slots, then the optimal power allocation is non-decreasing 
over slots. 

Proof: This follows immediately from (|23l ) and property PI, which implies that TJ < T k if 7/ < -y k 
for I < k. ■ 
An example that illustrates Corollary |3]is given in Fig. |3] where we see that the inverse of the SNR is 
non-increasing. It is easy to see that the converse of Corollary |3] is not true in general. That is, if the SNR 
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slot k 

1 ► 

Fig. 3. Structure of optimal power allocation Tjt with full SI and infinite 73 ma x, with increasing SNR 7*,, i.e, decreasing j^ 1 , over slot k. 
In this case, T£ must increase over slot k. 

is non-increasing, then the optimal power allocation may not be non-increasing over slots (in particular 
for the slot immediately after the TS). In conventional water-filling, however, both Corollary |3] and its 
converse hold, i.e., the optimal power allocation is non-decreasing over slots if and only if the SNR is 
non-decreasing over slots. 

We give an intuitive understanding of Corollary |3] and why the converse does not hold, to shed some 
light on how the energy harvesting constraints lead to a different optimal power allocation. First, let us 
consider the AWGN channel where the SNR is constant over slots. If all the harvested energy is already 
available in the first slot, i.e., there is only a single sum-power constraint, a uniform power allocation is 
optimal for the AWGN channel. However, in an energy harvesting system, maintaining a uniform power 
allocation may not be always possible due to the causal arrival of the harvested energy. Due to this non- 
uniform availability of harvested energy over slots, more energy only becomes available for transmission 
in the latter slots. Intuitively, we also expect more energy to be allocated for transmission in the latter 
slots such that the energy harvesting constraints in (119bl ) are satisfied. This type of strategy is optimal 
from Corollary |3] which applies since the SNR is constant and hence also non-decreasing. Next, consider 
the case where the SNR is non-decreasing over slots. From the water- filling algorithm under a single 
sum-power constraint, to achieve the maximum throughput it is optimal to allocate more power to the 
latter slots that have higher SNRs. This is consistent with the earlier observation that more power should 
be allocated to the latter slots such that the constraints in (|19b| ) are satisfied. Hence, it is also optimal to 
allocate more power to the latter slots. In general if the SNR is arbitrary, however, the high-SNR slots may 
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not correspond to the latter slots; hence intuitively the converse of Corollary [3] may not hold in general. 

2) Efficient Implementation: Based on the structural properties, we now develop an efficient algorithm 
to implement the staircase water- filling. 

Some definitions are in order. For convenience, let to = 0. We refer to the zth slot interval, where i = 
1, • • • , |<S|, as the slots between the ith and (i + l)th TS, specifically in the TS set Si = {U^i + 1, • • • , U}. 
Thus, [J i Si = K, and Si fl Sj = for i ^ j. The optimal set of TSs corresponding to an optimal power 
allocation is denoted as S* = {t\, t^, ■ • ■ , t* s *\} 

Corollary 4: The optimal power allocation performs staircase water- filling as follows: for every ith 
slot interval, where i = 1, • • • ,\S*\, conventional water- filling is performed subject to the sum power 
constraint of P(i) = J^fces Hk-i* where we denote H = B\ for notational simplicity. 

Proof: From property P2, all the harvested energy available in the ith slot interval, namely P(i), 
is used during the ith slot interval. This follows by induction for i = 1, • • • , |«S|. Moreover, the optimal 
power allocation in (1231 ) is equivalent to conventional water- filling. To maximize throughput, the optimal 
power allocation must then be to use conventional water-filling with sum power constraint of P(i) for 
every ith slot interval. ■ 

From Corollary |4] without loss of optimality the staircase water-filling solution comprises of multiple 
conventional water-filling solutions, one for each slot interval. The original optimization problem (Q~9) can 
thus be reduced to a search for the optimal TS set S* that has a size from 1 to at most K: 

T = max max 1% F {tf,P{l)) + 73&f(t£ +1) P(2)) 

+ + 7^(7^,'-!, P(|5|)) (24) 

subject to the power allocation P(l), • • • , P(|«S|) satisfying the constraints in (|19b| ). A brute force search 
based on (1241) is of a high computational complexity. Nevertheless, it turns out that it is optimal to simply 
employ a forward-search procedure, starting with the search of the optimal t*, then of the optimal t%, and 
so on until the last optimal TS t* 5 *| equals K, at which point the optimal size is also obtained. 

The first optimal TS t* can be found in Lemma|2]given below; by induction, the search of the subsequent 
optimal TSs will follow similarly. Lemma [2] requires the following feasible-search procedure for a given 
optimization problem (fT9l ): 

1) Initialize S x as an empty set. 
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2) For ti = 1, • ■ • , K, obtain the optimal power allocation from slot 1 to slot t\ by using a water-filling 
algorithm (such as Algorithm [T) assuming that all harvested energy is available, i.e., the sum power 
constraint is P max = B 1 + Y?i=i Hi. 

3) Admit t\ in the set S\ if the corresponding optimal power allocation satisfies the constraint (1 19b I 
for k — 1, • • • , t\. 

The set S 1 is non-empty; it contains at least the element ti = 1, as the constraint (|19bl ) in Step [3] is 
equivalent to the sum power constraint in Step |2] Moreover, the set Si includes all possible candidates 
for the optimal t\; the only candidates that are not included are those where at least one of the constraint 
in (1 1 9bb is not satisfied for slot k — 1, ■ • • , t\. 

Lemma 2: Let S\ be the feasible set of ti obtained by the feasible- search procedure. Then the optimal 
TS is given by the largest element in S\, i.e., 

= maxti. (25) 
tieSi 

Proof: If |«Si| = 1, then that only t\ must be optimal. Henceforth, assume \Si\ > 2. Consider two TSs 
t',t" G <Si, where t' < t" . Denote their respective optimal WLs obtained from the water- filling algorithm 
as z/, v". Then v' > v" . Otherwise if v' < v", more power is allocated for each time slot k = 1 ■ ■ ■ , t', if 
the WL v" is used, compared to the case where the WL v' is used. But since the power allocation with 
WL v' has used all available power at slot t' due to property P2 in Theorem |3] the power allocation with 
WL v" is infeasible and thus cannot be optimal. 

We now show that t' cannot be the optimal t\ by contradiction. Suppose that t\ = t', i.e., water-filling 
is used from slot 1 to slot t' with WL v' . The WL for the power allocation must then subsequently 
decrease at some slot t' < k < t", otherwise the sum power allocated from slots 1 to slot t" will be more 
than the sum power allocated with the (constant) WL v" , which violates the sum power constraint. But 
from property PI in Theorem |3j the optimal WL is non-decreasing. Thus, t\ ^ t' by contradiction. By 
induction, all elements in «Si, except for the largest one, are suboptimal. The only candidate left, namely 
the largest element, must then be optimal. ■ 

We now propose Algorithm |2] below to solve (|24) . which is optimal according to Theorem 0] below. 
Briefly, Algorithm |2] performs a forward-search procedure (starting from slot 1) in each of the outer 
iteration for t*, t^,--- , until tf 5 *, = K. Given that , t*_ x is found, to obtain t*, an inner iteration is 
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performed via a backward-search procedure, starting from slot t L , t x — , until slot 1. 



input : slot size K; SNRs {7/0}; harvested power {Hk\ where we let H = B\\ tolerance e 
output: optimal set of TSs S* = {t*} 

II initialization 
to = 0; 

for % — 1, 2, ■ • • , K do 

// Outer iteration: find t\, then t\, and so on 
for k = K, K - 1, • • • , U-i + 1 do 

// Inner iteration: find the largest feasible tj in (12 51) 

Use Algorithm \T\ for slot + 1 to slot k with inputs 

(i) SNRs {jt^+x,--- ,1k} 

(H) Pmax = ELL #i 

(iii) tolerance e, 

to give output {T* - ,T*}; 

if {Tj*, • • • , T^} satisfy the constraints dl9b\) then 
t* := fc; 

end 

end 

if t* = K then 
exit; 

end 

end 

Algorithm 2: Finding optimal TSs 

Theorem 4: Algorithm |2] obtains the optimal S* that solves the optimization problem in (l24l) or equiv- 
alently (TT9l) . 

Proof: The 2th outer iteration of Algorithm |2] finds the optimal t*. Consider z = 1. From Lemma |2] 
we can determine the optimal t* by finding the largest feasible t\. Without loss of optimality, we can 
modify the feasible-search procedure such that the search (in Step El) starts from the largest slot index to 
the smallest, and (Step [3]) terminates to give the optimal t* once a feasible t\ is found. These modifications 
lead to the inner iteration in Algorithm |2] 

From Property P2 in Theorem |3j in the first TS interval all the power available would be used. Since no 
power is available for subsequent slot intervals given t\, the power allocation for subsequent slot intervals 
can be optimized independent of the actual power allocated in the first slot interval. The throughput 
maximization problem from slot t* + l onwards can be solved similarly as before (after removing time 
slots 1, • • • ,t{). Thus, we apply the inner iteration again to determine t* 2 , similarly for t\ and so on, as 
reflected in the outer iteration of Algorithm |2] The iteration ends when the optimal TS equals K, which 
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is the largest possible value as stated in optimization problem in (1241) . ■ 
3) Update Algorithm when New Slots Become Available: Suppose that we have obtained the optimal 
S* based on Algorithm |2] for a J^-slot system. Now, a new slot becomes available for our use where 
its SI is known. We wish to obtain the new optimal solution for this (K + l)-slot system, say S* ew = 
0*new5*2new> ' ' ' }■ Instead of implementing Algorithm [2] afresh, we can obtain an update of <S* ew from 
S* as follows: 

• Consider i — 1 in the outer iteration of Algorithm |2] We only need to execute k = K + 1 in the 
inner iteration. If the constraints (119b| ) are satisfied, then we have obtained t\ new = K + 1, hence 
^new = {^inew} an d Algorithm |2] terminates. Otherwise, since we already know that t\ in the largest 
element in S* , we obtain immediately t* new = t\. 

• The subsequent iterations are executed similarly. For the ith outer iteration, where i — 2, 3, ■ • • , we 
only execute k = K+ 1 in the inner iteration. If the constraints (|19bl ) are satisfied, then t* new = K + l 
and Algorithm |2] terminates; otherwise i* = t*. 

Hence, for every outer iteration, only one additional inner iteration is executed until the constraints (1 19b I ) 
are satisfied. Since there are at most K outer iterations, we need to execute at most K inner iterations in 
total. In cases when the number of available slots can increase dynamically in a multi-user system, say 
when other users give up their slots and is assigned to our energy harvesting system, the above proposed 
update algorithm allows an efficient way to update S*. 

VI. Numerical Results 

To obtain numerical results, we assume that the SNR 7^. and the harvested energy H k are i.i.d. over 
time slot k and the channel is either an AWGN or Rayleigh fading channel, as described in Section IIII-DI 
We assume the initial stored energy B\ and the harvested energy Hu takes a value in {0, 0.5, 1} with equal 
probability and B max — > 00. To measure the performance of various schemes, we plotted the throughput 
per slot, i.e., the sum throughput divided by the number of slots K, as the average SNR 7 is increased. 

If causal SI is available, the optimal policy is obtained recursively by applying Lemma [T] Specifically, 
we first obtain Jk{B) in © via the closed-form results in Section ITlI-Di we drop the 7 and H arguments 
due to the i.i.d. assumption. Then we obtain Jk-i(i, B) in (|8bl , say by an iterative bisection method. The 
throughput are averaged over 10 4 independent realizations of 7 and H to give Jk-i(B). This procedure is 
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performed for different B, discretized in step size of 0.01, and stored to be used for the next recursion. The 
iteration is repeated for k — K — 2, • • • , 1. If instead full SI is available, the optimal policy is obtained 
by Algorithm |2j we have verified that our proposed algorithm is significantly faster but is equivalent 
to solving the problem via a standard optimization software. The throughput per slot is obtained from 
averaging the results from 10 4 independent Monte Carlo runs. 

The results are shown in Fig |4]for AWGN channels and in Fig. [5] for Rayleigh fading channels. The 
throughput in both cases, when either full SI or causal SI is available, is the same for K = 1, because 
any SI cannot be exploited for future slots. However, in both cases the throughput per slot increases as K 
increases. The increment is more substantial when full SI is available, intuitively because the SI can then 
be much better exploited. The incremental improvement as K increases is significant when K is small, 
but becomes less significant when K is large. The throughput with either full SI or causal SI does not 
differ significantly, possibly because the SI that can be further exploited from full SI is limited in our 
i.i.d. scenario. 

A. Heuristic Schemes with Causal SI 

Next, we consider two heuristic schemes that use causal SI yet can be easily implemented in practice, 
namely the naive scheme and the power-halving scheme. 

In the naive scheme, all stored energy Bk is used in every slot k, i.e., T/. = Bk. This is equivalent to the 
case of K = 1 in our optimization problem regardless of whether causal SI is available (see Lemma [U) or 
full SI is available (see Theorem |4]). In both cases, it is optimal to use all stored energy. As seen earlier, 
the case of K — 1 performs significantly worse than the optimal schemes for K > 2 in both cases. To 
obtain further improvement in the per-slot throughput, we need to further exploit the causal SI available. 

In the power-halving scheme, all stored power is used for transmission in the last slot, while for all 
other slots half of the stored energy is used, i.e., T k = WkB k where = 1 if k — K and = 1/2 
otherwise. This scheme is simple to implement. Intuitively, the present throughput is traded equally with 
the future throughput by splitting the battery energy B k into two halves. We note that this scheme also 
implicitly exploits causal information of the harvested energy (which accumulates as the stored energy 
Bk). Moreover, the power-halving scheme satisfies the following characteristics and thus likely improves 
the throughput when causal SI is only available: 
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Fig. 4. AWGN channel: optimal throughput when causal SI (blue with "x" markers) or full SI (red with "o" markers) is available for 



1) Tk increases with Bk, in accordance with Theorem |2] in the causal SI case. 

2) From Remark |2l if K = 2, the optimal power allocation is given by half of the battery energy Bi/2 
and a correction term that depends on the SNRs and harvested energy. Ignoring this correction term 
(which is not known due to the lack of full SI) then leads to the power-halving scheme for K = 2. 

3) More stored energy is deferred to be used in the latter slots, thus resembling the optimal policy 
with staircase WLs in the full SI case. 

Fig. [6] shows the throughput per slot obtained by averaging the numerical results from 2x 10 4 independent 
runs of Monte Carlo simulations, for both AWGN channels and Rayleigh fading channels. We fix the 
SNR at 20 dB. As benchmarks, we also plot the optimal throughput when full SI is available. This is 
because the computational complexity in solving the Bellman's equations in Lemma \T\ when causal SI is 
available becomes prohibitive for large K; moreover we observed earlier that the performance with causal 
SI is close to the performance with full SI. The results show that the power-halving scheme is able to 
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Fig. 5. Fading channel: optimal throughput when causal SI (blue with "x" markers) or full SI (red with "o" markers) is available for 
K = 1,2,4. 

improve on the per- slot throughput as K is increased, and is only within about 0.2 bits away from the 
case when full SI is available. Moreover, we see that at small K, the gap is even closer, as suggested 
by the second characteristic mentioned above. Similar results are obtained at lower SNR, with an even 
smaller throughput degradation compared to the case when full SI is available. Further performance gain 
may also be obtained by optimizing this tradeoff by considering the channel conditions explicitly. 



VII. Conclusion 

We considered a communication system where the energy available for transmission varies from slot to 
slot, depending on how much energy is harvested from the environment and expended for transmission in 
the previous slot. We studied the problem of maximizing the throughput via power allocation over a finite 
horizon of K slots, given either causal SI or full SI. We obtained structural results for the optimal power 
allocation in both cases, which allows us to obtain efficient computation of the optimal throughput. Finally, 
we proposed a heuristic scheme where numerical results show that the throughput per slot increases as 
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Fig. 6. Throughput based on the power-halving scheme. The optimal throughput when full SI is also plotted for comparison. 



K increases and performs relatively well compared to a naive scheme. 



Appendix A 
Proof of Theorem [H 

With 7, H fixed, we prove by induction that Jk{l, H, B) and Jk+i(j,H, x) are concave in B and x, 
respectively, for decreasing k = K,- ■ ■ ,1. 

Consider k 6 {1, • • ■ , K—l}. Suppose that Jk+i(j, H, B) is concave in B. We note that Jfc + i(7, H, min{_B max , x+ 
H}) is concave in x, as it is the minimum of Jfc+i(7, H, B max ) (a constant independent of x) and the 
concave function J k+1 (-f, H, x + H). It follows that J k+1 ('j,H, x) is concave in x, since expectation 
preserves concavity. From (|8bl , J k is a supremal convolution of two concave functions in B, namely / and 
Jk+i (with 7, H fixed). It follows that J k is concave in B, since the infimal convolution of convex functions 
is convex Q2] Theorem 5.4]. To complete the proof by induction, we note that Jk{i, H, B) = 1(7, B) is 
concave in B by assumption on the mutual information function /(•,•)• 
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Appendix B 
Proof of Theorem [2] 

We need Lemma [3] to prove Theorem |2] 

Lemma 3: Consider T*(B) = argmaxF(S, T), where the maximization is over interval TJ(£>) < 
T < T U (B) that depends on B. If Ti(B),T u (B) are non-decreasing in B, and if F has non-decreasing 
differences in (B,T), i.e., VT" >T,B' > B, 

F(B', T) - F(B, T) > F(B', T) - F(B, T), (26) 

then the maximal and minimal selections of T*(B), denoted as T(B),T(B), are non-decreasing. 

Proof: See proof in lfT3l Theorem 2]. ■ 
We now prove Theorem |2] with 7, i/ fixed; we drop these arguments from all functions. From (l8al . the 
optimal transmission power is T^(B) = B, which is increasing in B. We now apply Lemma [3] to establish 
that Theorem |2] hold for k < K. Let F(B,T) = I(T) + J k +i(B - T), according to @. Let T t (B) = 0, 
T U (B) = B, which are non-decreasing in B. To apply Lemma [3j it is sufficient to show that each term 
in F has non-decreasing differences in (B,T). Since I(T) is independent of B, trivially I(T) has non- 
decreasing differences in (B,T). To show that g(B — T) = Jk+\(B — T) has non-decreasing differences 
in (B,T), we note that g(y + 5) — g(y) < g(x + 5) — g(x) for x < y, 5 > 0, since g(x) = J k+l (x) is 
concave in x from Theorem [T] Substituting x = B — X" ,y = B — T, 5 = B' — B, we then obtain (|26|) with 
F(B,T) = g[B — T). From Theorem [[] the objective function in dHJ is concave, thus T*(B) is unique. 
From Lemma |3l T*(B) is thus non-decreasing in B, k E /C. 

Appendix C 
Proof of Corollary [2] 

Since K = 2 and full SI is available, from ©, (US we get 

71(71,51)= max #(T), (27) 

o<r<_Bi 

where the objective function is given by g(T) = log 2 (l + 71 T) + log 2 (l + 72 min{S max , B\ — T + 

Suppose ifi > -B max . Then min{_B max , B 1 — T + H 1 } = B max given T < B\. The optimal T that solves 
(1271) is then 

T* = B 1 ifH> B max . (28) 
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' max ? 



Suppose H x < B msx . Assume that < T < B 1 + Hi - B max . Then min{ J B max , B 1 -T + Hi} = B IS 
and so the optimal T to maximize g{T) subject to < T < Bi + Hi — -E> max is given by the largest value 
in the variable space, namely B x -\- H x — B m£LX . Thus, in general the optimal solution in (127b satisfies 
T* > [.Bi + Hi — £> max ] + , where = max(0, x). Without loss of generality, we can thus express T* as 

Tt = arg max g(T) (29) 

[Bi+Hi-B max ]+<T<Bi 

if < B max . Now if T > [S 1 +fT 1 -S In «] + , we have g(T) = log 2 (l+ 7l T)+log 2 (l+7 2 (B 1 -T+H x )), 
which is differentiable and concave. Observe that T in (TF71 ) solves the equation p'(T) = 0, i.e., T is the 
optimal solution for the unconstrained optimization problem maxg(T). By concavity of g(T), we can 
then obtain (T29l as 

T-,* = arg max fi'(^) 

[Bi+i?i-B max ]+<T<Bi 

Bi, f > B i; 

T, [fii + Hi - B max }+ <f<Bi- 

[B 1 + Hi — B miLX ] + , T < [Bi + Hi — B max ] + 
if Hi < B max . By re-writing the above conditions in terms of B x and combining the result with (128T ). we 
then obtain T* as stated in Corollary |2] 
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